# Open World

GameGen-X is a diffusion model specifically designed for generating and interactively controlling open-world game videos. The model achieves high-quality, open-domain video generation by simulating various features of game engines, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive control capabilities that allow it to predict and alter future content based on current video segments, simulating gameplay. To realize this vision, we meticulously collected and constructed an open-world video game dataset (OGameData) from scratch. This dataset is the first and largest of its kind for open-world video generation and control, comprising over a million diversified game video clips from more than 150 games, all equipped with informative subtitles powered by GPT-4o. GameGen-X underwent a two-phase training process, consisting of foundational model pre-training and instruction tuning. Initially, the model was pre-trained using text-to-video generation and video continuation methods, equipping it with the capability to generate long sequences of high-quality open-domain game videos. To further enhance its interactive control abilities, we developed InstructNet, which integrates expert multimodal control signals relevant to gaming. This allows the model to adjust latent representations according to user input, unifying character interaction and scene content control in video generation for the first time. During the instruction tuning phase, only InstructNet was updated while the pre-trained foundational model remained static, ensuring that the integration of interactive control capabilities did not compromise the diversity and quality of generated video content. GameGen-X represents a significant leap in video game design using generative models, demonstrating the potential of these models as complementary tools to traditional rendering techniques, effectively combining creative generation with interactive abilities.

GameGen-O is the first diffusion transformation model customized for generating open-world video games. By simulating various features of game engines, such as innovative characters, dynamic environments, complex actions, and diverse events, it enables high-quality, open-domain generation. Additionally, it offers interactive controllability, which allows for gameplay simulation. The development of GameGen-O involved extensive data collection and processing from the ground up, including the construction of the first open-world video game dataset (OGameData) and efficient sorting, scoring, filtering, and decoupling of titles through a proprietary data pipeline. This robust and comprehensive OGameData serves as the foundation for the model training process.

Grounding DINO 1.5, developed by IDEA Research, is a series of advanced models designed to push the boundaries of open-world object detection technology. The series includes two models: Grounding DINO 1.5 Pro and Grounding DINO 1.5 Edge, optimized for diverse applications and edge computing scenarios, respectively.

LARP is a language intelligent agent framework designed for role-playing in open world games. It includes cognitive architecture, environmental interaction, and character shaping modules, which allow for the creation of unique characters with distinct backgrounds and personalities, enhancing user interaction with intelligent agents. LARP facilitates flexible problem-solving through refined interaction and coherent long-term memory, helping language models adapt to the complexities of open worlds.